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J  This  report  summarizes  the  results  of  a  four  year  research  effort.  The 
primary  research  topic  addressed  is  automatic  shape  recognition  of  silhouettes 
or  contours.  Methods  studied  are  Fourier  descriptors,  Walsh  descriptors,  and 
2-D  moments.  Also  the  recognition  of  partially  correct  shapes  are  addressed. 
Three  methods  were  developed  to  solve  such  problems.  These  methods  are 
(1)  polygonal  fitting  of  a  boundary  followed  by  local  matching  of  distances 
and  angles;  (2)  Fourier  descriptor  representation  of  boundary  segments  followed 
by  dynamic  programming  to  match  segments  (dynamic  programming  is  modified  to 
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forms  using  Mellin  Transform  techniques. 


Other  research  topics  addressed  by  this  summary  are  ( 1 )  registration 
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cision  measurements  in  digital  images. 
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Foreword 

This  final  report  summarizes  the  results  of  a  four-year  research  effort  undertaken 
by  the  School  of  Electrical  Engineering  at  Purdue  University.  The  primary  purpose  of 
this  work  has  been  to  investigate  the  various  aspects  of  shape  recognition  in  digitized 
images.  As  indicated  in  detail  in  Section  ID,  five  Ph.D  theses  and  five  M.S.E.E.  theses 
resulted  from  this  research,  as  well  as  over  20  additional  publications.  The  author  is 
indebted  to  the  12  graduate  students  listed  in  Section  III  who  provided  the  program¬ 
ming  and  many  of  the  good  ideas  presented  in  these  publications. 

The  value  of  the  research  resulting  from  this  grant  is  difficult  to  estimate  without 
the  perspective  of  the  future.  However  the  results  of  experimental  tests  and  the 
increase  in  interest  by  other  investigators  indicate  that  the  global  shape  descriptors 
(especially  Fourier  descriptors  and  2-D  moments)  have  a  future  in  target  recognition 
systems  and  other  computer  visioin  applications.  Of  particular  interest  are  the  tech¬ 
niques  developed  for  partial  shape  recognition.  These  results  are  very  new  and  their 
usefulness  in  practical  applications  remains  to  be  seen.  Although  conference  presenta¬ 
tions  describing  these  results  have  been  made,  the  journal  publications  have  not  yet 
appeared  (some  have  not  yet  been  submitted).  However  the  potential  applications 
appear  significant.  The  work  on  the  Mellin  Transform  for  scale  invariant  recognition  of 
partially  correct  waveforms  and  shapes  appears  to  have  many  applications  in  fields  such 
as  acoustic  processing,  radar,  and  video  processing. 

The  author  would  like  to  thank  Dr.  Frank  Kuhl  of  the  U.  S.  Army  ARDC  for  his 
interest,  support,  and  research  contributions  to  this  project  and  Dr.  William  Sander  of 
the  U.  S.  Army  Research  Office  for  his  support  and  assistance. 
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I.  Statement  of  the  Problem  Studied 

The  primary  research  topic  addressed  by  this  effort  was  shape  recognition;  i.e., 
given  a  silhouette  or  contour  extracted  by  computer  from  a  2-D  digitized  image,  com¬ 
pare  the  shape  to  known  specific  shapes  to  recognize  the  unknown  shape.  The  libraries 
used  for  comparison  could  be  generaed  by  3-D  solid  models,  pictures  of  known  shapes, 
or  a  generic  description  of  a  shape  to  be  identified.  During  the  course  of  this  research, 
several  methods  were  compared,  improvements  were  added,  and  new  methods  were 
developed.  Emphasis  was  placed  on  recognition  of  aircraft  shapes.  Shape  recognition 
under  three  types  of  conditions  was  studied:  (1)  low  noise,  complete  detailed  contours; 
(2)  noisy  but  complete  contours;  and  (3)  incomplete  or  partially  correct  contours. 

Other  related  research  topics  were  addressed  during  the  course  of  this  project. 
These  topics  were  (1)  registration  and  segmentation  of  moving  objects;  (2)  measurement 
and  recognition  using  3-D  range  data;  (3)  image  compression  using  motion  detection; 
and  (4)  precision  measurements  in  digital  images. 
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ll.  Summary  of  the  Most  Important  Results 

A.  Development  of  Fourier  Descriptors,  2-D  Moments,  and  Walsh  Descrip¬ 
tors  for  Global  Shape  Recognition  and  the  Comparison  of  these  Methods. 

All  three  of  these  shape  description  methods  are  useful  for  recognizing  boundaries 
or  silhouettes  which  have  moderate  to  complex  detail  and  low  to  moderate  noise.  In 
the  Ph.D.  thesis  of  Groganfl],  these  three  methods  are  described  in  detail  and  comparis¬ 
ons  of  the  shape  discrimination  capabilities  of  each  are  described.  In  the  Ph.D.  thesis  of 
Rostampour[2],  improvements  in  the  2-D  moment  method  are  described  which  improve 
performance  considerably.  The  results  of  these  combined  studies  can  be  summarized  as 
follows: 

1.  For  recognition  of  low-noise,  high-detail  silhouettes  and  comparison  to  hun¬ 
dreds  of  possible  shapes,  Fourier  descriptors  performed  the  best,  Walsh  descrip¬ 
tors  were  faster  but  slightly  less  accurate,  and  2-D  moments  were  slightly  less 
accurate. 

2.  For  higher  noise  situations  (S/N  <  10  dB),  the  normalized  2-D  moments  per¬ 
formed  the  best.  However,  the  performance  of  all  methods  tested  fell  off  rapidly 
below  a  signal  to  noise  ratio  in  the  original  image  of  10  dB. 

3.  An  unknown  image  target  resolution  of  64x64  pixels  is  required  for  optimum 
classification  accuracy;  a  resolution  of  16x16  pixels  is  usable  but  classification 
accuracy  drops. 

4.  About  32  shape  features  must  be  calculated  for  best  accuracy  but  moderate 
accuracy  is  obtainable  with  only  12  features. 

5.  Classification  accuracy  drops  by  10%  when  10%  of  the  contour  is  incorrect  and 
drops  by  40%  when  20%  of  the  contour  is  incorrect. 

A  demonstration  video  tape  has  been  produced  which  shows  the  shape 
classification  performance  of  the  Fourier  descriptor  algorithms  using  video  imagery  of 
commercial  aircraft  filmed  at  O’Hare  airport.  A  description  of  the  system  used  can  be 
found  in  [5j  and  [6j. 

A  modular  shape  analysis  software  system  has  been  writted  in  Fortran  77  (some 
programs  have  also  been  rewritten  in  ‘C’)  to  allow  comparison  of  various  shape  descrip¬ 
tors.  This  system  has  modules  for  library  graphics  generation,  shape  descriptor  genera¬ 
tion  (Fourier  descriptors,  Walsh  descriptors,  and  2-D  moments),  shape  descriptor  nor¬ 
malization,  redundancy  reduction  transformation,  feature  vector  manipulation  (to  allow 
combinations  of  various  features),  and  classification  of  unknown  shapes. 

Other  publications  which  describe  details  and  applications  of  these  shape  recogni¬ 
tion  techniques  are  [11],  [15],  [16],  [17],  [18],  [19],  [21],  [24],  and  [26]. 
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B.  Accuracy  of  Fourier  Descriptors  for  Estimating  Orientation  of  a  3-D 
Object  from  TV  Images 

Since  the  Fourier  descriptor  can  be  used  to  compare  a  complex  shape  with  many 
very  similar  library  entries,  it  is  a  good  method  for  estimating  the  orientation  of  a  3-D 
object  with  respect  to  a  TV  camera.  The  method  used  involves  storing  descriptors  to 
find  the  best  match  with  an  unknown.  Interpolation  can  be  used  to  improve  the  accu¬ 
racy  even  further.  In  one  detailed  study  conducted  for  Battelle  Columbus  labs  [32],  the 
orientation  of  an  aircraft  could  be  determined  to  within  an  error  of  0.020  radians  for 
yaw,  0.023  radians  for  pitch,  and  0.011  radians  for  roll  (these  numbers  represent  one 
standard  deviation).  These  results  were  also  reported  at  an  Army  Research  Office 
Workshop  [32]. 


C.  Development  of  a  Method  to  Estimate  the  Required  Number  and  Aspect 
Angles  of  Library  Entries  for  Recognition  of  a  3-D  Object  from  any  Angle. 

The  number  and  direction  of  viewing  angles  required  to  recognize  a  3-D  object 
depends  on  the  object  itself.  If  memory  required  or  processing  speed  is  important,  it 
may  be  necessary  to  store  a  minimum  number  of  library  entries.  We  have  developed  a 
procedure  of  ordering  the  library  entries,  eliminating  redundant  views,  and  storing  the 
entries  differentially  which  requires  less  storage,  is  faster,  and  gives  accurate 
classification  results  [5],  [7],  and  [11]. 


D.  Recognition  of  Partial  Contours  using  Polygonal  Fitting  and  Local  Match¬ 
ing 


A  method  of  partial  contour  recognition  has  been  developed  which  involves  match¬ 
ing  sections  of  the  boundary  with  straight  line  segments  (polygonal  matching). 
Emphasis  is  placed  on  speed  and  repeatability  of  the  line  fitting.  A  binary  tree  decom¬ 
position  of  the  boundary  is  used.  The  features  extracted  are  straight  line  distances  and 
angle  changes  along  the  contour.  A  recognition  algorithm  has  been  programmed  which 
finds  the  best  match  between  the  unknown  and  straight  line  contours  independently  of 
scale,  orientation,  and  partially  incorrect  data.  The  recognition  algorithm  also  allows 
feature  combinations  so  that  shapes  extracted  at  different  resolutions  (which  could  have 
a  different  number  of  line  segments)  can  still  be  matched  [9],  [27]. 


E.  Recognition  of  Partial  Contours  using  Dynamic  Programming  to  Match 
Segments  Described  with  Fourier  Descriptors. 

This  method  involves  segmenting  a  contour  into  segments  and  describing  each  seg¬ 
ment  with  Fourier  descriptors.  When  two  contours  are  to  be  compared,  the  distances 
between  each  of  their  respective  segment  sets  may  be  viewed  as  an  inter-segment  dis¬ 
tance  table.  A  dynamic  programming  approach  for  this  problem  has  been  developed 
which  finds  a  path  through  this  distance  table  which  gives  the  best  local  matches 
independent  of  any  few  bad  matches  along  the  path.  Thus  the  normal  dynamic  pro¬ 
gramming  approach  which  results  in  a  global  optimum  path  has  been  modified  to  find  a 
large  number  of  local  optimums  independent  of  whether  it  results  in  a  global  optimum. 
Tests  on  partial  aircraft  contours  have  shown  that  this  method  works  very  well  [31]. 

F.  Recognition  of  Partial  Matches  using  Meilin  Transform  Techniques 

Tracing  the  boundary  of  an  object  distorted  by  noise,  obscuration,  or  poor  segmen¬ 
tation  produces  a  boundary  function  with  an  unknown  scaling  factor  when  the  range  is 
unknown.  Because  of  this  unknown  scaling,  we  cannot  use  available  signal  processing 
techniques  which  are  primarily  concerned  with  a  class  of  signals  with  identical  scale  fac¬ 
tors  after  some  normalization  procedure,  e.g.,  the  methods  of  Fourier  descriptors, 
moments,  matched  filtering,  and  Wiener  filtering.  The  Meilin  transform  has  a  property 
which  enables  us  to  calculate  these  unknown  scale  factors.  We  have  shown  that  there 
exists  a  homomorphic  system  which  converts  scale  information  into  translation  infor¬ 
mation,  or  the  Meilin  transform  into  the  Fourier  transform  and  this  fact,  in  turn, 
enables  us  to  use  the  existing  signal  processing  techniques.  Digital  implementation  of 
the  homomorphic  system  and  the  Meilin  transform  have  been  studied  based  on  the  Mei¬ 
lin  sampling  theorem  derived  in  this  research  and  on  finite  dimensional  function  spaces, 
which  alleviates  the  problem  of  the  aliasing  effect  of  the  digital  Meilin  transform  which 
is  present  in  existing  algorithms.  To  solve  the  reported  problems  of  the  Meilin 
transform  such  as  lower  domain  emphasis,  boundary  effects,  and  a  problem  related  to 
changes  in  amplitude  of  a  signal,  we  have  introduced  the  generalized  Meilin  transform 
and  shown  that,  with  the  orthonormal  Meilin  transform,  all  these  problems  are  allevi¬ 
ated. 

A  fast  direct  Meilin  transform  technique  has  been  developed  during  the  course  of 
this  research.  The  basic  features  of  this  approach  are  that  two  partially  matching 
waveforms  can  be  compared  and  the  time-axis  scaling  factor  which  gives  the  best 
match  can  be  calculated  (useful  for  passive  range  detection)  and  that  features  can  be 
calculated  from  waveforms  which  are  invariant  to  time-axis  scaling  of  the  waveforms 
(useful  for  partial  shape  recognition  and  processing  of  acoustic  signals  derived  from 


rotating  machinery  of  unknown  speeds). 

Initial  partial  shape  recognition  experiments  using  the  Mellin  Transform  are  given 
in  [1],  [20],  and  [24].  Fundamental  developments  in  Mellin  Transform  theory  and  its 
application  to  signal  processing  problems  is  given  in  [4].  This  work  is  very  new  and  has 
not  yet  been  submitted  for  journal  publication. 

G.  Algorithms  for  Registration  and  Segmentation  of  Moving  Objects 

Another  project  has  been  devoted  to  the  analysis  of  sequential  video  frames  for 
registration,  motion  detection,  temporal  enhancement,  and  moving  object  segmentation. 
A  registration  technique  has  been  developed  based  on  binarization  and  correlation 
which  allows  rapid  sequential  frame  alignment.  After  the  imagery  is  registered,  any 
motion  in  the  frame  can  be  detection  by  measuring  temporal  variance  at  each  point. 
Each  moving  object  can  then  be  registered  and  temporally  enhanced  by  temporal 
median  filtering.  This  results  in  sharp  clear  pictures  of  moving  objects  even  when  any 
single  frame  is  noisy  and  blurred. 

After  background  registration,  temporal  variance  measurements  over  sequential 
frames  are  used  to  provide  moving  area  masks.  When  sequential  moving  area  masks 
are  ANDed,  the  moving  objects  are  segmented.  Also  the  background  surrounding  the 
moving  area  mask  can  be  used  to  closely  estimate  the  background  region  statistics  for 
target  segmentation  [2]. 

H.  Image  Compression  using  Block  Truncation  Coding 

Research  has  also  been  conducted  into  the  coding  of  image  data.  The  technique  of 
block  truncation  coding  has  been  extended  to  color  data  and  an  additional  modification 
has  been  made  which  allows  more  accurate  reconstructions  and  faster  computation. 
Real-time  compression  of  24-bit  color  imagery  to  2  bits/pixel  with  little  visible  degrada¬ 
tion  has  been  accomplished  [8], [13].  This  technique  has  also  been  extended  so  that  the 
temporal  correlation  of  sequential  data  can  be  used  to  reduce  the  bit  rate  even 
further[10].  This  work  has  application  in  remote  reconnaissance,  image  data  storage, 
and  teleconferencing. 
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I.  Precision  Target  Location  using  Fourier  Descriptors 

The  use  of  Fourier  descriptors  for  shape  recognition  was  applied  to  another  prob¬ 
lem  of  interest  to  photogrammetrists.  In  a  separate  grant  by  the  U.  S.  Army  Research 
Office  (DAAG29-81-K-0063),  the  Fourier  descriptors  and  2-D  moments  developed  under 
this  grant  were  applied  to  the  problem  of  precision  measurements  in  digitized  imagery. 
It  was  found  that  pointing  accuracies  better  than  1/10  of  a  pixel  can  be  obtained  in 
many  situations  (e.g.,  pointing  at  the  center  of  a  cross  shaped  object  in  a  noisy  back¬ 
ground).  This  results  of  this  work  are  reported  in  the  Final  Report  (Sept.  1984)  and  in 
the  references  [12]  and  [32]. 


J.  Automatic  Interpretation  of  Range  Data 

The  aim  of  this  research  program  is  to  develop  methods  for  recognizing  objects 
from  their  range  maps.  During  the  past,  shape  recognition  algorithms  were  based 
mostly  on  silhouette  and/or  line  representation  of  objects.  However,  silhouettes  and 
lines  cannot  be  extracted  with  great  reliability  because  of  the  difficulties  caused  by 
reflectance  variability,  shape  and  shadow  effects.  With  range  data,  boundaries  and 
edges  are  defined  as  range  and  range  gradient  discontinuities,  these  being  immune  to 
the  problems  with  reflectance  data. 

In  our  program  we  devised  strategies  in  which  objects  are  characterized  on  the 
basis  of  the  orientation  of  their  major  surfaces;  these  orientations  being  obtained  by 
fitting  planes  and  other  regular  curved  surfaces  to  measured  range  maps.  We  have  also 
developed  a  new  scheme  for  extracting  edges  from  the  range  data.  This  scheme  consists 
of  first  constructing  a  synthetic  image  from  the  range  map  in  which  the  gray  level  at 
each  point  is  proportional  to  the  orientation  of  the  local  surface  normal  and  then  apply¬ 
ing  a  reflectance  data  edge  detector  to  this  synthetic  image.  In  [21]  we  show  examples 
of  both  the  surface  extraction  and  the  detection  of  edges  with  the  new  procedure. 
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in.  Publications  and  Technical  Reports 

The  following  list  of  publications  represents  work  resulting  entirely  or  in  part  from 
this  grant.  This  section  also  serves  as  the  list  of  references  for  Section  II. 


Theses  Completed  by  Graduate  Research  Assistants  (All  from  School  of  Electri¬ 
cal  Engineering,  Purdue  University,  West  Lafayette,  Indiana,  Prof.  O.  R.  Mitchell, 

Thesis  Advisor). 
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